Add support for custom scripts #1056

Lucca-mito · 2024-10-13T23:23:12Z

Summary

First-class support for adding custom scripts to a DocC-generated website. These may be local scripts, in which case the website will continue to work offline, or they may be external scripts at a user-specified URL. This support is in the form of a custom-scripts.json file, the scripting analog of theme-settings.json.

Full pitch: https://forums.swift.org/t/pitch-support-for-custom-scripts-in-docc/75357.

Dependencies

Corresponding change in swift-docc-render.

Testing

Unit tests added.
To test the feature end-to-end, you'll need to use the version of swift-docc-render with the corresponding changes. Follow the steps in the pitch for adding any custom scripts you want to your documentation website (your own scripts, or a library such as MathJax or D2). If you want a specific example to test the implementation:
- Copy the custom-scripts.json and the custom mathjax-config.js script (shown in the pitch) to your documentation catalog.
- Add LaTeX expressions to your in-source documentation.
- docc convert the documentation catalog with your custom scripts. Observe that the modified swift-docc copied custom-scripts.json and the custom scripts into the documentation archive.
- Preview the documentation archive. Observe that the LaTeX expressions are rendered on the page.
Live demo: see here and here. The LaTeX expressions in the documentation text are dynamically rendered into equations via custom scripts.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

Added tests
Ran the ./bin/test script and it succeeded
Updated documentation if necessary
- Existing documentation does not need to change. New documentation for this feature will be added after the design and implementation are finalized.

d-ronnqvist · 2024-10-16T10:29:08Z

Sources/SwiftDocC/Infrastructure/DocumentationBundle.swift

-    @available(*, deprecated, renamed: "init(info:baseURL:symbolGraphURLs:markupURLs:miscResourceURLs:customHeader:customFooter:themeSettings:)", message: "Use 'init(info:baseURL:symbolGraphURLs:markupURLs:miscResourceURLs:customHeader:customFooter:themeSettings:)' instead. This deprecated API will be removed after 6.1 is released")
-    public init(
-        info: Info,
-        baseURL: URL = URL(string: "/")!,
-        attributedCodeListings: [String: AttributedCodeListing] = [:],
-        symbolGraphURLs: [URL],
-        markupURLs: [URL],
-        miscResourceURLs: [URL],
-        customHeader: URL? = nil,
-        customFooter: URL? = nil,
-        themeSettings: URL? = nil
-    ) {
-        self.init(info: info, baseURL: baseURL, symbolGraphURLs: symbolGraphURLs, markupURLs: markupURLs, miscResourceURLs: miscResourceURLs, customHeader: customHeader, customFooter: customFooter, themeSettings: themeSettings)
-        self.attributedCodeListings = attributedCodeListings
-    }
-


This is a source breaking change. Like the deprecation message says; we can't remove this until after 6.1 is released. See the "Introducing source breaking changes" section of the contributions guidelines

Hey @d-ronnqvist,

Thank you for the comments! And apologies for the delayed response.

I agree that I should not remove the deprecated init in this PR: in truth, I only removed the deprecated init in this first revision to ask for help in the PR discussion for how to best avoid making this breaking change. I’m unable to build swift-docc when both versions of the init exist; this happens only with my changes, and I’m not sure why.

To explain the issue I’m facing, consider as an example the following call to DocumentationBundle.init in ConvertService+DataProvider.swift:

bundles.append( DocumentationBundle( info: info, symbolGraphURLs: symbolGraphURLs, markupURLs: markupFileURLs, miscResourceURLs: miscResourceURLs ) )

The only difference between the deprecated DocumentationBundle init and the non-deprecated init is that the deprecated init has an attributedCodeListings parameter. That parameter is optional, so in theory, the call above could be either to the non-deprecated init or to deprecated init. However, the compiler disambiguates the call by prioritizing the non-deprecated init.

But when I added customScripts as an optional parameter to the non-deprecated init — and changed the deprecated init to pass customScripts: nil to the non-deprecated init — I got an “Ambiguous use of 'init'” error at the call above.

But why? Surely if the compiler could disambiguate the init call before I added the customScripts parameter (by prioritizing the non-deprecated init) then it should be able to do the same after I added customScripts, no? Maybe the compiler’s disambiguation rules are more nuanced than I realize.

Regardless, unless there’s a better solution to this “Ambiguous use of 'init'” problem I'll just manually disambiguate the calls by explicitly specifying customScripts: nil where appropriate. I can also add a comment clarifying that the explicit customScripts: nil can be removed after Swift 6.1 (when the deprecated init is removed), like so:

bundles.append( DocumentationBundle( info: info, symbolGraphURLs: symbolGraphURLs, markupURLs: markupFileURLs, miscResourceURLs: miscResourceURLs, customScripts: nil // Explicit `customScripts: nil` can be removed after the deprecated `DocumentationBundle.init` is removed. ) )

If this would not be the best way to circumvent the init ambiguity, please let me know. Thanks!

IIRC you can add the @_disfavoredOverload attribute to the deprecated initializer to avoid the ambiguity.

d-ronnqvist

This is making an unintentional source breaking change that should be undone.

Additionally, we're in the process of migrating away from LocalFileSystemDataProvider so the new code to find the custom-scripts.json file also need to be added to DocumentationContext.InputProvider. There's already a test that verifies that both implementation discover the same files which would also be good to update so that it verifies the behavior for this new type of file.

d-ronnqvist · 2024-10-16T10:30:52Z

Sources/SwiftDocC/Infrastructure/DocumentationContext.swift

+    ///
+    /// - Parameter bundleIdentifier: The identifier of the bundle to return download assets for.
+    /// - Returns: A list of all the custom scripts for the given bundle.
+    public func registeredCustomScripts(forBundleID bundleIdentifier: BundleIdentifier) -> [DataAsset] {


I would prefer if this was package level access for now so that we don't have to make more source breaking changes to change it to not have a bundle ID parameter when #1059 lands.

Sounds good!

d-ronnqvist · 2024-10-16T10:32:44Z

Tests/SwiftDocCUtilitiesTests/ConvertActionTests.swift

@@ -3251,6 +3251,7 @@ class ConvertActionTests: XCTestCase {

        XCTAssertEqual(fileSystem.dump(subHierarchyFrom: targetURL.path), """
        Output.doccarchive/
+        ├─ custom-scripts/


Question: is it expected that we create this folder even if the user didn't pass any custom scripts?

Yes it is! Every subfolder of a documentation archive is created even when they are no files to add to the subfolder.

d-ronnqvist · 2024-10-16T10:42:02Z

Sources/SwiftDocC/Infrastructure/Workspace/LocalFileSystemDataProvider+BundleDiscovery.swift

FYI, the method of input discovery in this file is about to be deprecated in #1059 and won't be used by DocC anymore.

These same changes need to be implemented in DocumentationContext.InputProvider which is what DocC will use to discover its inputs after #1057 lands.

It would also be good to update DocumentationInputsProviderTests.testDiscoversSameFilesAsPreviousImplementation() to verify that both implementations discover the custom scripts file the same. (This can happen before either of those PRs are merged)

d-ronnqvist · 2024-10-16T10:45:34Z

Sources/SwiftDocC/Infrastructure/DocumentationBundle.swift

        self.rootReference = ResolvedTopicReference(bundleIdentifier: info.identifier, path: "/", sourceLanguage: .swift)
        self.documentationRootReference = ResolvedTopicReference(bundleIdentifier: info.identifier, path: NodeURLGenerator.Path.documentationFolder, sourceLanguage: .swift)
        self.tutorialsRootReference = ResolvedTopicReference(bundleIdentifier: info.identifier, path: NodeURLGenerator.Path.tutorialsFolder, sourceLanguage: .swift)
        self.technologyTutorialsRootReference = tutorialsRootReference.appendingPath(urlReadablePath(info.displayName))
        self.articlesDocumentationRootReference = documentationRootReference.appendingPath(urlReadablePath(info.displayName))
    }

-    @available(*, deprecated, renamed: "init(info:baseURL:symbolGraphURLs:markupURLs:miscResourceURLs:customHeader:customFooter:themeSettings:)", message: "Use 'init(info:baseURL:symbolGraphURLs:markupURLs:miscResourceURLs:customHeader:customFooter:themeSettings:)' instead. This deprecated API will be removed after 6.1 is released")


FIY: Since the name of the replacement changed with the new parameter you may need to update the renamed portion of this deprecation annotation. Even if it's not needed, it's nice to do it since it enables the fixit for any code that still uses the deprecated API.

d-ronnqvist · 2025-02-20T08:40:30Z

Sources/SwiftDocCUtilities/Action/Actions/Convert/ConvertFileWritingConsumer.swift

@@ -120,6 +120,18 @@ struct ConvertFileWritingConsumer: ConvertOutputConsumer {
        for downloadAsset in context.registeredDownloadsAssets(forBundleID: bundleIdentifier) {
            try copyAsset(downloadAsset, to: downloadsDirectory)
        }
+
+        // Create custom scripts directory if needed. Do not append the bundle identifier.


Why not append the bundle ID here?

Add support for custom scripts

949508f

Lucca-mito mentioned this pull request Oct 13, 2024

Add support for custom scripts swiftlang/swift-docc-render#903

Open

3 tasks

d-ronnqvist reviewed Oct 16, 2024

View reviewed changes

d-ronnqvist requested changes Oct 16, 2024

View reviewed changes

d-ronnqvist added the source breaking DocC's public API isn't source compatible with earlier versions label Oct 16, 2024

N3v1 mentioned this pull request Oct 23, 2024

Simplyfiying the implementation of custom content (Support for Custom HTML) #1072

Open

d-ronnqvist reviewed Feb 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for custom scripts #1056

Add support for custom scripts #1056

Lucca-mito commented Oct 13, 2024 •

edited

Loading

d-ronnqvist Oct 16, 2024

Lucca-mito Feb 20, 2025 •

edited

Loading

d-ronnqvist Feb 21, 2025

d-ronnqvist left a comment

d-ronnqvist Oct 16, 2024

Lucca-mito Feb 20, 2025

d-ronnqvist Oct 16, 2024

Lucca-mito Feb 20, 2025

d-ronnqvist Oct 16, 2024

d-ronnqvist Oct 16, 2024

d-ronnqvist Feb 20, 2025

Add support for custom scripts #1056

Are you sure you want to change the base?

Add support for custom scripts #1056

Conversation

Lucca-mito commented Oct 13, 2024 • edited Loading

Summary

Dependencies

Testing

Checklist

Choose a reason for hiding this comment

Lucca-mito Feb 20, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

d-ronnqvist left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lucca-mito commented Oct 13, 2024 •

edited

Loading

Lucca-mito Feb 20, 2025 •

edited

Loading